Regarding focus detection apparatuses for detecting a focus state of an image-forming optical system, Japanese Patent Laid-Open No. 4-267211 discloses an apparatus that performs so-called pupil division focus detection using a two-dimensional image sensor (image pickup element) having a micro-lens formed in each of two-dimensionally arranged pixels. In the image pickup element used in the apparatus disclosed in Japanese Patent Laid-Open No. 4-267211, the micro-lens and a photoelectric converting part in each pixel are relatively displaced. An image pickup apparatus provided with this focus detection apparatus adds signals from the pixels whose relative displacement directions of the micro-lens and the photoelectric converting part are mutually different to produce a normal captured image. On the other hand, the image pickup apparatus performs correlation calculation on paired image signals produced by pixels groups whose relative displacement directions of the micro-lens and the photoelectric converting part are mutually different to obtain a phase difference between the paired image signals, and calculates a defocus amount of an image-forming optical system (image-taking optical system) on the basis of the phase difference.
Moreover, vignetting of light fluxes forming paired optical images to be used for the focus detection, which is caused by a light-blocking member such as an optical element holding member that holds an optical element constituting part of the image-forming optical system, causes asymmetry in each of paired image signals corresponding to the paired optical images (that is, decrease of a degree of coincidence between the paired image signals). Japanese Patent Laid-Open No. 5-127074 discloses a focus detection apparatus that transforms a specific image restoration filter stored in an image pickup apparatus according to an aperture ratio of an image-forming optical system, an exit pupil position thereof or a phase difference amount, and applies the transformed image restoration filter to image signals, thereby performing focus detection with high accuracy. The application of the image restoration filter is performed by convolution of the image signals with the image restoration filter, so that a pitch of the image restoration filter coincides with that of the image signal.
However, the focus detection apparatus disclosed in Japanese Patent Laid-Open No. 5-127074 fixes the pitch of the image restoration filter and a number of the image restoration filters applied to the image signal regardless of the defocus amount and an aperture value of the imaging-forming optical system. Thus, the focus detection apparatus cannot perform accurate image restoration due to aliasing distortion in a near-in-focus state where a filter length becomes long, which deteriorates focus detection accuracy.